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A Video Camera Utilizing Sequential Diversity Imaging For Image Clarification 

Background of the Invention 

The present invention relates to imaging through a random medium, such as the 
turbulent atmosphere or the slowly changing characteristics of a video camera. 

Wavefront sensing is employed to determine the distorting wavefront caused by 
the turbulent atmosphere and one or more the following methods are generally 
employed to provide an estimate of the distorting wavefront. . 

1. Dithering. This method of wavefront sensing continuously changes the 
adaptive optics and monitors the image quality of the observed image. US 
Patents 3,979,585 entitled "Adaptive imaging telescope with camera-computer 
transform image quality sensing and electro-optic phase shifting " and 5,146,073 
entitled "Linear wavefront sensor camera with deformable mirror for recording 
velocity compensated images" and the publication by M. A. Voronstov and V. P. 
Sivokon, entitled "Stochastic parallel-radiant-descent technique for high- 
resolution wave-front phase-distortion correction," J. Opt.Soc.Am.,A,15,2745 
(1988) are examples of this approach. They are sequential techniques, in the 
sense that a physical property of the adaptive optics is changed, and if the next 
(time-sequential) image is sharper the change is increased, otherwise the 
change is reversed. This is a physical search for "best focus" and is how the 
human eye and most cameras work. 

2, Shearing Interferometer. This method of wavefront sensing uses a reference 
beam to create an interference pattern, from which the unknown wavefront is 
estimated. It requires a laser-based interferometer. US patent 5,923,400 entitled 
"Real-time film animation technique" describes this approach. 
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3.Shack-Hartmann Sensor. This device employs an array of lenses to focus multiple, small 
images, each seen through a different section of the aperture, onto a detector. Shifts in the 
small images are caused by local tilts in the waveform, which allows the wavefront to be 
reconstructed. US Patents 4,141 ,652 entitled "Sensor system for detecting wavefront 
5 distortion in a return beam of light" and 5,350,91 1 entitled "Wavefront error estimation 

derived from observation of arbitrary unknown extended scenes" disclose the use a Shack- 
Hartmann sensor. 

4. Curvature Sensing. This method is described in the publication by F. Roddier et 
al.entitled "A simple low-order adaptive optics system for near-infrared applications," 

10 Publications of the Astronomical Society of the Pacific, 1 03, 1 31 (1 991 ), whereby two or 
more images are measured along the path of the optical system. The local curvature of the 
propagating wave is determined and it is propagated, by computer calculation, back to the 
aperture to form the wavefront estimate. 

5. Phase diversity. This method employs diverse images, measured simultaneously 

15 whereby the diversity is a quadratic phase shift, which can be introduced by defocusing the 
optical system. Additional equipment in needed to record the out-of-focus image. US 
Patents 4,309,602,entitled "Wavefront sensing by phase retrieval", 5,384,455 entitled 
"Measurement-diverse speckle imaging", 5,610,707 entitled "Wavefront sensor for a staring 
imager", 6,107,617 entitled " Liquid crystal active optics correction for large space based 

20 optical systems ", and the publication by R. Paxman, et al., entitled "Optical misalignment 
sensing and image reconstruction using phase diversity," J. Opt. Soc. Am., A, 5, 914 (1988) 
each use phase diversity to estimate the wavefront. It is noted that the phase diversity 
approach to wavefront sensing was used to determine the aberration in the Hubble 
telescope project. 

25 Each of the aforementioned methods of wave front sensing requires additional equipment 
and functions to control adaptive optics in the camera system. 

One purpose of the instant invention is to provide a system for determining aberrations 
within a video camera caused by the medium and to eliminate the aberrations using existing 
30 camera focusing equipment with no additional equipment or additional time-consuming 
searches . 
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Summary of the Invention 



The invention utilizes interactive sequential diversity imaging within a video camera for 
5 determining the aberrations caused by the medium and controls an adaptive optic in the 
optical system to eliminate the aberrations. Such interactive diversity imaging uses one or 
more of the images, each with a known diversity such as phase, wavelength, or spatial shift, 
to deduce both the unknown object and the parameters of the medium. When images of the 
object are recorded as a video sequence and when an adaptive optics system is used to 
10 sharpen the images, the changes in the adaptive optics, per se, are used as the diversities. 
The sequential frames comprise the diversity images and a standard sequential processor is 
employed to control the adaptive optic. 
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Brief Description of the Drawings 



Figure 1 is a diagrammatic representation of the imaging system, which employs an 
adaptive optic and a sequential diversity processor within a video camera, in accordance 
with the invention; 

Figure 2 is a flow chart representation of the sequential diversity algorithm used within the 
20 sequential diversity processor of Figure 1 ; and 

Figure 3 depicts a computer simulation of 6 frames of a video sequence achieved within the 
video camera of Figure 1. 



Description of the Preferred Embodiment 



The sequential diversity imaging system 12 within the video camera 13 depicted in phantom 
in Figurel , receives an optical signal 1 which may have aberrations introduced by a 
randomly changing medium. The optical signal passes through the camera aperture 2 and is 
imaged by a lens 3. Between lens 3 and detector 5 is an Adaptive Optic 4, hereinafter, "AO, 
30 such as described in the publication entitled "Phase retrieval and diversity in adaptive 
optics," R.A. Gonsalves Opt. Eng., 21, 829 (1982). 
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The detector 5 is an array of photodiodes, not shown, each of which provides a signal of 
magnitude related to the intensity of radiant energy incident thereon. The AO is controlled to 
reduce the wavefront error signal as described in the aforementioned US patent 4,309,602. 
The output of detector 5 is a video sequence of digital images, as indicated at 6. This 
5 sequence of images is the input to a Sequential Diversity Processor 7 which produces 
control signals 8 to control the configuration of the AO, so as to cancel the aberrations 
introduced by the random medium. The AO could comprise a high-resolution device to allow 
a wide range of correction mechanisms, such as Zernike polynomial fitting of a complicated 
wavefront, or the automatic focusing and registration mechanisms used in consumer 
10 camcorders. 

Figure 2 depicts a flow diagram depicting the image flow through the lens 3, AO 4, detector 
5 and the processing of data within the Sequential Diversity Processor 7 as shown in Figure 
1 . The k th frame of the video output 6, l(k), is an input to the Diversity Algorithm 10. Other 

15 inputs to the Diversity Algorithm 10 are successively delayed versions of l(k), namely l(k-1), 
l(k-2), which are delayed and stored in a digital buffer within the Sequential Diversity 
Processor 7 of Figure 1 along with the Delays 9A-9C as depicted in Figure 2. 
The k* output of the Diversity Algorithm, D(k), and delayed versions thereof, are used as 
inputs to the Predicted Algorithm 1 1 which predicts the change in the random medium for 

20 the next frame of the video sequence and controls the AO 4 with a control signal T(k) as 
indicated at 8. The other inputs to the Predicted Algorithm 1 1 are delayed versions T(k-1), 
T(k-2)— . of the control signal T(k). 

To further describe the flow diagram of Figure 2, the following notations assume that the 
media aberration is due to an unknown wavefront distortion, which is typical of atmospheric 
25 distortion. The calculations within the Diversity Algorithm 1 0 of Figure 2 employ the 
following:. 

W(k)= Unknown distorting wavefront at time k. 
T(k) = Phase on the AO at time k. 
C(k)= Compensate phase to be estimated 
30 = W(k) +T(k). (1) 

l(k) = Measured image at time k. 
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D(k) = Diversity phase. 
With l(k-1) as the first image and l(k) as the diversity image, the diversity phase Dk is the 
change in the AO phase from time k-1 to time k, such that: 

D(k)- T(k)-T(k-1). (2) 

5 

To estimate the residual phase, let C(k) represent the estimate Q(k) from equation (1), 
Q(k) = W1(k) + T(k), (3) 

where W1(k) is an estimate of W(k), the unknown phase at time k. Since T(k) is known, at 
10 time k + 1 the AO phase is set to the negative of the unknown distorting wavefront W at time 
k+1 . An estimate of the distorting wavefront at time k, W1 (k), provides a good estimate of 
W(k+1) when AO updates are within the time constant of the changing medium. Therefore: 

T(k+1) = -W1(k), (4) 
which tends to cancel the wavefront distortion at k+1 . 
IS Solving (3) for W1(k) and substituting into equation (4), results in 
T(k+1) = - Q(k) +T(k) .which implies 

T(k) = -Q(k-1) +T(k-1). (5) 
For the diversity phase Dk insert equation (5) into equation (2) whereby 
D(k)= ( - Q(k-1)+ T(k-1))- T(k-1)= - Q(k-1) 
20 The new diversity D(k+1) is now defined 

D(k+1) = -Q(k). (6) 
Which provides the diversity at time k+1 . 

Inserting equation (6) into equation (5) results in or the new AO phase: 

T(k+1) = T(k)+D(k+1). (7) 
25 Equations (6) and (7) result in the interactive diversity imaging function in accordance with 
the invention. 

In summary, the sequential diversity algorithm 10 and predicted algorithm 11 of Figure 2 
contained within the sequential diversity processor 7 of Figure 1 employ the diversity D(k) 
and the diverse images, l(k) and l(k-1), to calculate an estimate of the phase, Q(k), which is 
30 the sum of the wavefront distortion plus the AO setting, T(k). The new diversity, D(k+1), is 
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set to the negative of Q(k), according to equation (6), and it is added to the current AO 
phase to form the new AO phase, T(k+1), according to equation (7). 



As shown in Fig. 2, the inputs to the diversity algorithm 10 are the measured image l(k) at 
5 time k; the diverse images l(k-1), l(k-2) via delays 9A, and the diversities D(k-1), D(k-2) via 
delays 9B. The inputs to the predicted algorithm 1 1 are the estimated phase diversity Dk; 
and the phases T(k-1), T(k-2) via delays 9C. 

Figure 3 depicts a computer simulation of frames 1-6 as achieved within the sequential 
10 diversity processor 7 within the video camera 1 3 of Figure 1 . The object 17 comprises four 
lines of text viewed through a turbulent atmosphere 1 via the video camera lens 3. The 
original unprocessed images 15 of the text are shown in lower regions of each frame and 
the processed images 16 are shown in the upper regions thereof. Comparing the original 
images 15 to the processed images 16 within each frame depicts how the processed 
15 images are clear and readable within 6 frames. The Strehl ratio, the ratio of the peak value 
of the unprocessed Point Spread Function, "PSF" to the peak value of the processed PSF 
as described in the aforementioned publication "Phase retrieval and diversity in adaptive 
optics," wherein higher Strehl ratio defines higher image quality is now employed to 
determine image quality for the original unprocessed images 15 and the processed images 
20 16. For the 6 original unprocessed images depicted in Fig. 3, the average Strehl ratio is 
0.09 whereas the average Strehl ratio for the processed images is 0.48, which results in an 
improvement of about a factor of 5. 

Accordingly, the invention teaches interactive sequential diversity imaging within a video 
25 camera for determining the aberrations caused by the medium and controls an adaptive 
optic in the optical system to eliminate the aberrations. 
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